Field of the Invention
The present invention relates to an image processing apparatus which corrects a deterioration of an image caused by an image pickup optical system.
Description of the Related Art
For an object captured via an image pickup optical system, light emitted from one point cannot be converged to another point and has a minute spread due to an influence of a diffraction, an aberration, or the like that occurs in the image pickup optical system. Such a minutely-spread distribution is referred to as a PSF (point spread function). Due to the influence of the image pickup optical system, the captured image is formed with the PSF convolved with the image, and accordingly, the image is blurred and its resolution is deteriorated.
Recently, the captured image is typically stored as electronic data, and a method (image restoration processing) of correcting the deterioration of the image caused by the image pickup optical system has been proposed. The deterioration state of the image varies depending on a position in the image.
Japanese Patent Laid-open No. 2013-25473 discloses an image processing method of performing optimum restoration processing depending on a deterioration state at each position in an image while reducing a calculation amount even when an image deterioration varies depending on the position according to a lens characteristic. The literature “OSA VOLUME 62, NUMBER 1 Jan. 1972 Bayesian-Based Iterative Method of Image Restoration” discloses an image restoration method by an iterative calculation using so-called Lucy-Richardson method.
When the number of iterations is not appropriately set in performing the image restoration processing by using the iterative calculation, the cost of calculation increases. Furthermore, in many cases, the number of iterations of calculation is a dominant factor with respect to the strength of the image restoration. For example, if the number of iterations of calculation is insufficient, the degree of the restoration is insufficient. On the contrary, if the number of iterations of calculation is too large, the degree of the restoration is excessive, which results in occurrence of a harmful effect such as ringing. Accordingly, it is necessary to set the number of iterations appropriately depending on characteristics of the image deterioration at a position or a region in the image. When the image restoration processing is performed with an appropriate strength, a method of terminating the iterative calculation processing based on a certain evaluation value is typically used.
When the image on which the restoration processing is to be performed is extremely large or the deterioration characteristic of the image varies depending on the position or the region in the image, it is difficult to terminate the iterative calculation processing appropriately by comparing the RMS (Root Mean Square) of an image which is not updated with the RMS of an updated image. When the deterioration characteristic of the image varies depending on the position or the region in the image, satisfactory correction cannot be performed only by controlling the number of iterations. In order to achieve appropriate image restoration, it is necessary to control a correction amount appropriately during each iterative calculation processing.
However, neither Japanese Patent Laid-open No. 2013-25473 nor the literature “OSA VOLUME 62, NUMBER 1 JANUARY 1972 Bayesian-Based Iterative Method of Image Restoration” describes these specific control methods. Accordingly, with respect to an arbitrary combination of an image pickup optical system and an image sensor, it is difficult to control a correction amount appropriately in the iterative calculation processing for a shift-variant deterioration in an image to perform appropriate image restoration processing for each position in the image.